Creating Incentives for Honest Rating Submission by Rating the Raters

Reputations of electronic parties are important for their economic viability in business settings. Potential clients of e-services cannot rely solely on their own experiences to gain knowledge about e-services’ reputations. Plainly aggregating rating comments from other clients can often be misleading. Therefore, we are presenting a reputation aggregation model which seeks for the subgroup of raters that (1) contains the largest degree of overall agreement and (2) derives the resulting reputation scores from their comments. We complement the reputation system by adding policies that promote both truth revelation for the rater commentary, and also competition over service quality between providers. Then we describe the recursive algorithm of the model, which judges a rater’s commenting quality based on the divergence between his and the other raters’ comments. This algorithm feeds the commenting quality back into the aggregation function as a weight on the rater’s comments. We evaluate our algorithm in simulations of three challenging threat scenarios. To this end, we show that our aggregation model can be effectively used to deselect weak rating accuracy, and to filter out a malicious collective of raters which makes up almost half of the rating population. Finally, we show that the model is able to reveal provider behavior that is biased in favor of specific groups of raters. 1 Motivation for a Reputation Reporting Service Clients choose a service provider on the basis of (1) price versus promised performance aspects such as, for example, a provider’s contractual terms in an SLAs and (2) the client’s confidence in how well this provider will deliver on the negotiated performance. Aggregating the confidence of many clients resembles the reputation of a provider. The importance of reputation in the clients’ decisions often is underestimated in the designs of electronic market places. This has been the case in, for example, Giovanetti and Ristuccia’s [6] analysis on the band-x backbone bandwidth market, where the researchers found that clients did not rely much on the reported performance numbers, but more so on the reputations of large, well-known providers. Introducing widely accepted reputation systems for eservices requires addressing a collection of pitfalls that are inherent to them. Most of these pitfalls are not of a technical nature. For technical policies such as “we want to keep commentators anonymous”, we are able to devise technical solutions. The questions that are difficult to address are in the nature of the design philosophy. For once, one needs a definition of the scales by which to measure the reputation of a provider a definition which includes the understanding of the pragmatic meaning of these scales. The reputation model we are presenting makes only one assumption about the choice of the reputation valuation function, namely that the reputation system operator has chosen an appropriate function. The question, then, arises: Can we trust the rating submissions from all raters? Should we give more weight to those raters who are seemingly more trustworthy than those who might be less well-informed? To address these questions, our model has the ability to redistribute the influence it assigns to a rater on the aggregated scores, and to do so in favor of certain better informed raters. In addition, we are able to produce different views of the reputation results, depending on which rater, or set of raters, we ex ante trust more. A separate threat model category exists on the side of the providers: Do they treat all their clients equally? Can we recognize provider discrimination? Can we distinguish discrimination behavior from biased client behavior? We address these different threat models by simulation them in challenging rating scenarios. Finally, do clients have some incentive to submit ratings at all? Do they have a tangible incentive state their experiences